NLlex – a tool to generate lexical analyzers for natural language

In this paper we present a natural language lexical analysis program generator (NLlex) that looks like Unix lex extended with morphological analysis and other Natural Language (NL) elements. NLlex generates a C program which is linked with a morphological analyzer and with other modules, in order to produce a NL processor. As a particular case, NLlex can generate modules to work: as a lexico-morphological analyzer (to be called from yacc, NLyacc, btyacc or any modules that need it) as a simple lexical processor tool NLlex can also deal with ,and be tuned to, the so frequently seen non textual elements (markup elements, LATEX like things, dates, quotes, ...) An interface between NLlex and Prolog have been developed and a Perl interface is under development.